The "Delete" button in the main VirtualBox window lets you remove a virtual machine which you no longer need. All settings for that machine will be lost. However, any hard disk images attached to the machine will be kept; you can delete those separately using the Disk Image Manager (described just above).

You cannot delete a machine which has snapshots or is in a saved state, so you must discard these first.